"The highlighted tokens are primarily function words, particles, and common connectors in Chinese (and some in Vietnamese and English), such as grammatical markers, conjunctions, and punctuation. These tokens are essential for sentence structure, logical flow, and connecting clauses or ideas, often marking relationships, transitions, or the boundaries of statements."
Score Type | Accuracy | Precision | Recall | F1 score | TPR | TNR | FPR | FNR |
---|---|---|---|---|---|---|---|---|
detection | 0.74 | 0.786 | 0.66 | 0.717 | 0.66 | 0.82 | 0.18 | 0.34 |
fuzz | 0.78 | 0.759 | 0.82 | 0.788 | 0.82 | 0.74 | 0.26 | 0.18 |